Astronomy's dirty window to space: A detailed map of dust in the Milky Way

When we observe distant celestial objects, there is a possible catch: Is that star I am observing really as reddish as it appears? Or does the star merely look reddish, since its light has had to travel through a cloud of cosmic dust to reach our telescope?